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I.  ABSTRACT  (CoMlnw  mi  nmM  MB*  H  t 


Electronic  equipment  training 

Generalized  maintenance  trainer  simulator  (GMTS) 


MMry  tBmMf  kf  wim 


Three  experiments  were  conducted  to  evaluate  the  effectiveness  of  the  generalized 
maintenance  trainer  simulator  (GMTS)  as  a  supplementary  device  for  laboratory 
troubleshooting  practice.  Students  using  the  GMTS  scored  as  well  on  final  tests  of 
performance  on  actual  equipment  as  did  students  who  had  trained  on  actual  equipment. 
It  was  concluded  that  GMTS  provides  effective  laboratory  troubleshooting  practice. 


FOREWORD 


This  research  was  performed  in  support  of  Navy  decision  coordinating  paper  Z0789, 
work  unit  PN.01  (Class  "A"  Electronic  Equipment  Maintenance  Training  System  (EEMT)). 
Its  mission  sponsor  is  the  Chief  of  Naval  Operations  (OP-01);  appropriation  sponsor,  the 
Chief  of  Naval  Operations  (OP-98);  and  requisition  sponsor,  the  Chief  of  Naval  Material 
(MAT-08E1). 

This  is  the  eighth  report  describing  the  generalized  maintenance  trainer  simulator 
(GMTS)  or  its  successor,  the  EEMT.  The  GMTS  had  its  genesis  in  prototype  systems 
developed  under  the  sponsorship  of  the  Office  of  Naval  Research  by  the  Behavioral 
Technology  Laboratories,  University  of  Southern  California.  The  contributions  of  the  late 
Dr.  Joseph  Rigney  and  of  Dr.  Douglas  M.  Towne  in  developing  the  concept  and  carrying 
the  development  forward  are  sincerely  appreciated. 

The  objective  of  the  effort  described  herein  was  to  determine  whether  the  GMTS 
performed  satisfactorily  in  the  laboratory  training  environment.  Of  the  previous  reports, 
two  described  GMTS  field  evaluations  and  hard  ware/ software  development  (NPRDC  TRs 
80-30  and  81-9);  three,  the  EEMT  concept  evaluation,  system  definition,  and  master  plan 
(NPRDC  TN  79-3,  TR  81-11,  and  SR  81-19);  and  two,  the  GMTS  and  the  process  of 
developing  a  data  base  for  training  (NPRDC  TN  82-6  and  SR  82-17). 
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Problem 


Operational  Requirement  50  of  Duly  1976,  which  defined  the  need  for  the  electronic 
equipment  maintenance  training  (EEMT)  system,  focused  on  the  problems  inherent  in  using 
actual  equipment  for  training  purposes,  the  need  to  Improve  maintenance  training  in  the 
Navy,  and  the  need  to  bridge  the  gap  between  basic  training  and  equipment-specific 
training.  One  approach  to  the  problem  inherent  in  maintenance  training  was  the 
development  of  a  maintenance  training  device  having  a  design  completely  independent  of 
the  design  of  the  equipment  being  simulated.  The  generalized  maintenance  training 
simulator  (GMTS)  was  designed  to  satisfy  this  requirement. 

Objective 


The  objective  of  this  effort  was  to  determine  if  the  GMTS  performed  satisfactorily  in 
the  laboratory  training  environment. 

Approach 

Four  GMTS  systems  were  installed  at  the  AN/WSC-3  school,  Advanced  Electronics 
Schools  Department,  Service  School  Command,  San  Diego.  A  test  of  the  trainers' 
effectiveness  was  performed  during  the  period  from  December  1980  to  September  1981. 
Three  experiments  were  conducted.  In  two  experiments,  an  experimental  group  used  the 
GMTS  while  a  control  group  used  actual  equipment.  In  the  third,  all  students  used  the 
GMTS  approximately  40  percent  of  the  time.  A  total  of  128  students  from  14  classes 
participated  in  the  three  experiments. 

Results 

Posttraining  troubleshooting  proficiency  measures  for  the  experimental  (trainer)  and 
control  (actual  equipment)  groups  were  not  statistically  different.  Times  required  for 
problem  solution  were  essentially  the  same.  Students  indicated,  however,  that  they 
preferred  using  the  actual  equipment. 

Conclusions 


The  results  indicate  that  the  trainers  are  an  effective  substitute  for  actual  equipment 
in  the  school  environment.  The  GMTS  provided  troubleshooting  practice  equivalent  in 
measured  effectiveness  to  that  provided  by  the  use  of  actual  equipment. 
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INTRODUCTION 


Problem  and  Background 


Informed  field  tests  showed  that  students  were  able  to  develop  troubleshooting 
proficiency  for  maintenance  tasks  after  practicing  with  the  generalized  maintenance 
training  simulator  (GMTS)  (Rigney,  Towne,  King,  &  Moran,  1978;  Rigney,  Towne,  Moran,  & 
Mishler,  1980).  GMTS  is  the  development  model  for  an  interactive,  image-based 
computer-controlled  trainer  simulator  designed  to  provide  simulated  "hands-on" 
experience  to  maintenance  technicians.  It  is  the  physical  embodiment  of  the  "Rigney 
trainer"  concept  rather  than  a  physical  entity  in  itself.  The  concept  is  that  of  an 
interactive  training  system  that  uses  two-dimensional  equipment  images  to  reflect  the 
state  of  the  equipment  being  simulated  (Lahey  &  Malec,  1981).  Data  bases  prepared  by 
subject  matter  experts  are  used  to  display  images  that  simulate  manipulation  of 
equipment  states  and  use  of  test  equipment  to  take  readings  of  equipment  parameters. 

Use  of  the  trainer  in  maintenance  (or  operator)  training  has  inherent  advantages  over 
the  use  of  actual  equipment.  It  will  usually  be  less  expensive  than  the  equipment  it 
simulates,  will  be  more  easily  used  and  maintained,  and  will  provide  training  that  is 
otherwise  impractical  because  of  hazards  to  personnel  or  equipment.  The  concept  is 
expected  to  have  wide  applications  in  the  field  of  electronic  maintenance  training,  with 
the  possibility  of  extension  into  other  fields  where  similar  problems  exist. 

Two  major  goals  are  reflected  in  the  design  of  the  GMTS  and  its  successor,  Unit  1  of 
the  electronic  equipment  maintenance  training  system  (EEMT;  Device  No.  11B106).  The 
first,  which  is  to  have  a  truly  generalized  trainer  capable  of  simulating  a  variety  of  very 
different  actual  equipments  without  requiring  computer  program  changes,  has  been 
substantively  realized  in  that  several  different  equipment  data  bases  have  been  developed, 
installed  in  the  trainers,  and  tested,  including  a  fleet  communications  system  (Rigney  et 
al.,  1978),  the  AN/SPA-66  radar  repeater  (Rigney  et  al.,  1980),  and  the  AN/WSC-3 
transceiver  system.  The  second  goal,  to  provide  efficient  training  that  meets  with 
student  and  instructor  acceptance,  is  addressed  herein. 

Objective 

The  objective  of  this  effort  was  to  determine  whether  the  GMTS  performed 
satisfactorily  in  the  laboratory  training  environment. 


APPROACH 

Four  GMTS  trainers  were  installed  in  the  AN/WSC-3  School,  Advanced  Electronics 
Schools  Department,  Service  School  Command,  San  Diego.  Following  an  equipment 
checkout  and  familiarization  phase,  which  began  in  the  spring  of  1980,  three  separate 
experiments  were  conducted  using  data  base  systems  for  the  AN/WSC-3  Satellite 
Communications  System  (ECHO  and  LOS  versions).  Civilian  students  were  excluded  from 
the  experiments.  These  experiments  are  described  below. 

Experiment  1 

Subjects 

Experiment  1  involved  all  of  the  military  students  (N  =  61)  in  seven  classes  (Nos. 
80014  to  81005,  held  from  August  1980  to  February  1981)  (see  Table  1).  Prior  to  starting 
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Table  1 


Distribution  of  Students 


Class 

Start  Date 

Experimental 

Group 

(N) 

Control 

Group 

(N) 

Total 

Experiment  1 

80014 

8/11/80 

3 

5 

8 

80015 

9/8/80 

6 

6 

12 

81001 

10/6/80 

3 

3 

6 

81002 

11/3/80 

5 

5 

10 

81003 

12/15/80 

5 

4 

9 

81004 

1/12/81 

5 

4 

9 

81005 

2/9/81 

3 

4 

7 

Total 

30 

31 

61 

Experiment  2 

81006 

3/9/81 

6 

5 

11 

81007 

4/6/81 

5 

6 

11 

81008 

5/4/81 

5 

6 

11 

Total 

16 

17 

33 

Experiment  3 


81009 

6/1/81 

— 

-- 

10 

81010 

7/6/81 

— 

— 

6 

81011 

8/3/81 

— 

— 

8 

81012 

9/8/81 

— 

10 

Total 

-- 

-- 

34 

the  laboratory  practice  periods,  the  instructors  matched  students  by  experience.  One 
member  of  each  pair  was  assigned  to  the  experimental  group;  and  the  other,  to  the  control 
group,  by  means  of  a  coin  toss.  In  classes  with  an  odd  number  of  military  students,  the 
"extra"  student  was  sometimes  assigned  to  the  experimental  group  and  sometimes  to  the 
control  group. 

Procedure 


Experimental  and  control  group  students  attended  classroom  lectures  together  and 
performed  all  job  sheet  assignments  on  actual  equipment.  Control  group  students  used 


actual  equipment  to  practice  the  ten  troubleshooting  practice  problems  included  in  the 
course,  while  experimental  students  used  the  GMTS  (ECHO  version  of  the  equipment)  for 
eight  practice  problems  and  actual  equipment  for  the  remaining  two.  No  remediation  was 
given  to  either  group  before  the  final  performance  test  in  each  class.  All  students  took 
the  final  performance  tests  on  actual  AN/WSC-3  equipment. 

During  both  practice  and  final  test  sessions,  the  course  instructors  kept  records  of  (1) 
the  time  required  by  each  student  to  complete  each  practice  problem  and  each  final 
performance  test  problem,  and  (2)  the  number  of  replacements  made  during  practice  and 
during  the  final  performance  tests.  Since  single-fault  problems  require  only  one 
replacement,  the  number  of  replacements  made  was  a  measure  of  errors.  A  student  who 
made  an  error  was  usually  informed  by  the  trainer  simulator  (experimental  group)  or  by 
the  instructor  (control  group)  that  the  replacement  was  wrong  and  was  then  allowed  to 
continue  the  problem.  If  the  student  made  three  errors  on  a  problem,  the  instructor 
usually  ended  the  problem  for  that  student.  A  replacement  number  of  "4"  was  used  to 
indicate  failure. 

After  finishing  the  laboratory  practice  periods,  students  in  each  group  completed  the 
appropriate  form  of  a  questionnaire  designed  to  assess  student  reaction  to  the  training 
(see  appendix). 

Analysis 

Performance  data  were  analyzed  using  one-way  analyses  of  variance  (BMDP2V, 
Dixon,  1981).  The  number  of  replacements  was  analyzed  using  a  chi-square  test 
(BMDP4F,  Dixon,  1981).  Questionnaire  data  were  analyzed  using  the  Mann-Whitney  test 
(Guilford  &  Fruchter,  1978,  pp.  218-221). 

Experiment  2 

Subjects 

Experiment  2  involved  all  of  the  military  students  (N  =  33)  in  three  classes  (Nos. 
81006,  81007,  and  81008,  held  from  March  to  May  1981)  (see  Table  1).  As  in  Experiment 
1,  students  were  randomly  assigned  to  the  experimental  and  control  groups  before 
practice  troubleshooting  began  in  each  class. 

Procedure 

The  procedure  was  generally  the  same  as  in  Experiment  1,  except  that  students  in  the 
experimental  group  did  all  ten  troubleshooting  practice  problems  on  the  trainer — eight  on 
the  ECHO  version  of  the  AN/WSC-3  and  two  on  the  LOS  version. 


During  Experiment  2,  students  were  allowed  to  make  only  one  replacement.  If  the 
student  replaced  the  wrong  module,  the  problem  was  terminated.  A  replacement  number 
of  "2"  was  used  to  indicate  failure. 


Analysis 


The  analyses  were  the  same  as  for  Experiment  1. 
Experiment  3 
Subjects 


Experiment  3  involved  till  the  military  students  (N  =  34)  in  four  classes  (Nos.  81009  to 
81012,  held  from  June  to  September,  1981)  (see  Table  1).  For  this  experiment,  students 
were  not  separated  into  experimental  and  control  groups. 

Procedure 


All  students  had  access  to  actual  equipment  or  to  the  trainers  for  troubleshooting 
practice.  Students  were  simply  assigned  to  either  the  trainer  or  the  actual  equipment  as 
one  or  the  other  became  available.  An  attempt  was  made  to  balance  the  time  students 
spent  on  the  trainer  or  on  actual  equipment  so  that  all  had  equal  GMTS  and  actual 
equipment  time.  However,  because  of  difficulties  with  both  the  actual  equipment  and  the 
trainers,  this  objective  was  not  always  achieved. 

The  experimental  group  version  of  the  opinion  questionnaire  was  administered  to 
students  in  two  classes  (Nos.  81009  and  81011).  A  modified  version  of  the  questionnaire, 
which  permitted  comparison,  was  administered  to  students  in  the  other  two  classes  (Nos. 
81010  and  81012). 

Analysis 

Since  there  was  no  experimental  treatment  effect,  the  only  within-experiment 
analysis  possible  was  a  within-group  analysis  of  the  response  to  two  questions  in  the 
modified  questionnaire— one  dealing  with  the  effectiveness  of  GMTS  (#2);  and  the  other, 
with  the  effectiveness  of  actual  equipment  (#3).  Also,  responses  made  by  Experiment  3 
students  to  key  questionnaire  items  and  their  performance  were  compared  to  the 
responses  and  performance  of  students  in  Experiments  1  and  2.  The  analyses  were 
otherwise  the  same  as  for  Experiment  1. 


RESULTS 

Table  2  presents  the  mean  practice  times,  final  performance  test  times,  and  number 
of  replacements.  Tables  3  and  4  respectively  present  a  breakdown  by  problems  of  mean 
performance  times  and  responses  to  certain  questionnaire  items.  (Differences  between 
Tables  2  and  3  are  caused  by  missing  data  effects  in  the  analyses  used.)  Results  for  each 
experiment  are  discussed  in  the  remainder  of  this  section. 

Experiment  1 

As  shown  in  Table  2,  there  were  no  significant  differences  in  time  required  to  solve 
problems  or  in  number  of  replacements.  There  was  some  interaction  between  groups  and 
problems  during  the  practice  performance  tests  (df  =  9,  468,  MSW  =  379.2,  F  =  2.01, 
p  <  .05),  as  is  indicated  in  Table  3.  The  problem  breakdown  in  Table  3  indicates  that, 
although  practice  times  varied  in  both  directions,  only  those  for  problems  1  and  5  differed 
significantly. 
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Table  2 

Mean  Problem  Completion  Times  and  Number  of  Replacements 


Experiment  1 

Experiment  2 

Experiment  3 

Exper. 

Control 

Exper. 

Control 

Total 

Item 

Group 

Group 

Group 

Group 

Group 

Practice  Problems: 

Time  required  (Min.) 
No.  Replacements3 

Final  Test  Problems: 

Time  required  (Min.) 
No.  Replacements3 


50.3 

50.1 

48.2 

46.8 

42.3 

9.5 

9.7 

10.3 

10.6 

10.1 

59.3 

59.7 

50.5 

52.3 

43.0 

11.2 

10.9 

8.6 

8.7 

9.2 

Note.  None  of  the  differences  are  significant. 

Based  on  10  practice  problems  and  8  final  test  problems. 
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Table  3 


Problem  Completion  Times  (Min.) 


Experiment  1 

Experiment  2 

Experiment  3 

— 

Problem 

! 

Exper. 

Group 

(N=30) 

Control 

Group 

(N=31) 

t  test 
(Exp.  vs. 
Cont.) 

Exper. 
Group 
(N  =  16) 

Control 
Group 
(N=  17) 

t  test 
(Exp.  vs. 
Cont.) 

Total 

Group 

(N=34) 

t  test  (Exp.  3 
Total  Group  vs. 
Exp.  2  Exp.  & 
Cont.  Groups) 

Practice 

1 

62.0 

48.7 

2.34** 

65.0 

48.6 

NS 

48.3 

-- 

2 

59.0 

69.5 

NS 

63.7 

56.7 

NS 

54.6 

— 

3 

54.3 

52.0 

NS 

44.6 

49.8 

NS 

50.3 

-- 

4 

51.0 

54.5 

NS 

40.2 

55.8 

2.11** 

42.3 

-- 

5 

44.9 

34.9 

2.30** 

44.5 

43.3 

NS 

35.0 

1 .76**  (Control) 
1.68**  (Exp.) 

6 

55.0 

47.5 

NS 

42.6 

42.1 

NS 

42.0 

-- 

7 

42.0 

43.5 

NS 

38.3 

46.1 

NS 

37.6 

— 

8 

38.8 

46.5 

NS 

28.1 

41.9 

2.81* 

40.7 

— 

9 

49. 7a 

54.0 

NS 

54.3 

51.9 

NS 

42.6 

— 

10 

43. 03 

42.4 

NS 

57.7 

53.9 

NS 

43.0 

-- 

Final 

1 

49.7 

46.2 

45.2 

49.7 

NS 

39.8 

— 

2 

55.1 

52.2 

NS 

46.7 

44.3 

NS 

58.6 

— 

3 

48.2 

54.1 

NS 

44.3 

42.6 

NS 

32.3 

1 .82**  (Control) 

■ 

* 

60.5 

57.2 

NS 

48.1 

55.6 

NS 

52.8 

— 

j 

5 

58.6 

55.6 

NS 

57.5 

52.1 

NS 

54.5 

— 

6 

66.2 

65.0 

NS 

57.1 

57.0 

NS 

40.4 

3.01*  (Control) 

7 

79.9 

80.5 

NS 

57.4 

69.8 

NS 

45.2 

3.26*  (Control) 

8 

— 

54.3 

45.5 

NS 

47.5 

47.5 

NS 

45.3 

1 

3 

Run  using  actual  equipment. 


*p  <  .01. 

**p  <  .05. 


i)  :  t 


I 


Table  6 


Responses  to  Questionnaires 


Experiment 

Experiment  2 

z 

(Comparison  of 

Experiment  3^ 

Exper. 

Control 

Exper. 

Control 

Exper.  Group 

Eva!,  of 

Eval.  of 

Group 

Croup 

z 

Group 

Group 

z 

Responses  in 

Trainer 

Act.  Equip.  z 

Question 

(N  =  30) 

(N=31) 

(N=  16) 

(N= 17) 

Exps.  1  &  2) 

Effec. 

Effec. 

1. 

Compared  to  other  training  (1-1) 
a.  Much  more  enjoyable 

0 

4 

2.60* 

1 

4 

3.2a* 

1.78** 

b.  More  enjoyable 

10 

15 

2 

7 

~ 

c.  About  as  enjoyable 

10 

11 

4 

6 

— 

~ 

d.  Less  enjoyable 

6 

1 

4 

0 

— 

— 

e.  Much  less  enjoyable 

2 

0 

5 

0 

f.  No  response 

2 

0 

0 

0 

— 

2. 

Leaming/practice  technique  (2-2) 

3.86* 

6.07* 

2.19** 

3.73* 

a.  Very  effective 

4 

13 

0 

6 

1 

10 

b.  Effective 

10 

17 

2 

9 

3 

4 

c.  Both  effective  and  Ineffective 

13 

0 

10 

2 

7 

i 

d.  Somewhat  Ineffective 

2 

0 

4 

0 

3 

0 

e.  Very  ineffective 

i 

0 

0 

0 

1 

0 

f.  No  response 

0 

1 

0 

0 

I 

1 

3. 

Trainer  equipment  easy  to  use  (3-3) 

NS 

2.20**  NS 

a.  Very  easy 

6 

7 

3 

6 

— 

-- 

b.  Easy 

17 

20 

6 

ii 

-- 

c.  Neither  easy  nor  difficult 

4 

4 

6 

0 

—• 

— 

d.  Difficult 

0 

0 

I 

0 

— 

-- 

e.  Very  difficult 

0 

0 

0 

0 

— 

-- 

f.  No  response 

3 

0 

0 

0 

— 

-- 

4. 

Setting  switches  (6-4) 
a.  Very  easy 

4 

16 

3.75* 

2 

9 

2.99* 

2.00** 

b.  Easy 

10 

13 

7 

8 

— 

-- 

c.  Neither  easy  nor  difficult 

9 

2 

3 

0 

— 

— 

d.  Difficult 

5 

0 

3 

0 

— 

-- 

e.  Very  difficult 

1 

0 

1 

0 

— 

-- 

f.  No  response 

1 

0 

0 

0 

~ 

•• 

5. 

Using  test  equipment  (7-5) 
a.  Very  easy 

8 

7 

NS 

7 

8 

NS 

NS 

b.  Easy 

11 

20 

5 

7 

— 

-- 

c.  Neither  easy  nor  difficult 

7 

4 

2 

2 

— 

-- 

d.  Difficult 

0 

0 

0 

0 

-- 

-- 

e.  Very  difficult 

0 

c 

0 

0 

-- 

f .  No  response 

4 

0 

2 

0 

“ 

— 

6. 

Problem  difficulty  (10-6) 
a.  Much  too  difficult 

0 

0 

NS 

0 

0 

NS 

NS 

b.  Too  difficult 

0 

0 

1 

0 

— 

-- 

c.  About  correct 

28 

30 

12 

15 

— 

— 

d.  Too  easy 

1 

1 

2 

1 

~ 

-- 

e.  Much  too  easy 

0 

0 

0 

0 

— 

-- 

f.  No  response 

1 

0 

i 

1 

— 

— 

7. 

Amount  of  practice  (1 1-7) 

NS 

1.87**  NS 

a.  Much  too  little 

I 

1 

i 

0 

— 

-- 

b.  Too  little 

11 

10 

6 

1 

— 

-- 

c.  About  right 

15 

15 

9 

16 

— 

-- 

d.  More  than  necessary 

0 

0 

0 

0 

-- 

e.  Much  more  than  necessary 

0 

0 

0 

0 

— 

— 

f.  No  response 

5 

0 

0 

•• 

8. 

Prior  familiarity  with  WSC-3  (13-8) 
a.  Very  familiar 

0 

0 

NS 

1 

0 

NS 

NS 

b.  Familiar 

1 

2 

1 

0 

— 

-- 

c.  Somewhat  familiar 

5 

4 

2 

5 

— 

--  • 

d.  Not  very  familiar 

5 

5 

3 

6 

~ 

-- 

e.  Not  at  all  familiar 

18 

16 

8 

6 

~ 

-- 

f.  No  response 

1 

4 

1 

0 

•• 

“Numbers  In  parentheses  refer  to  Item  numbers  In  experimental  and  control  questionnaires  respectively. 

Comparison  of  responses  of  students  In  classes  81010  and  81012  (N  *  16)  to  questionnaire  Items  concerning  effectiveness  of  trainer  and  actual 
equipment. 

*p  <  .01. 

**p  <  .05. 


DISCUSSION 


For  training  purposes,  operational  military  electronics  systems  are  usually  more 
expensive,  more  easily  damaged,  and  more  hazardous  than  are  simulator  trainers. 
Further,  the  introduction  of  problems  in  actual  equipment  is  time  consuming  and  can 
seriously  degrade  the  unit's  durability  and  maintainability.  Trainer  simulators  can  be  used 
very  effectively  to  improve  training  where  one  or  more  of  these  conditions  holds  true.  If 
the  actual  equipment  is  very  expensive,  a  training  facility  that  can  afford  only  one  or  very 
few  actual  units  could  provide  significantly  more  troubleshooting  practice  by  installing  a 
number  of  trainers  to  support  use  of  the  actual  equipment. 

The  experiments  reported  herein  address  the  question  of  whether  using  a  trainer 
simulator  like  the  GMTS  to  support  hands-on  electronic  maintenance  training  is  an 
effective  alternative  to  the  use  of  actual  equipment.  The  results  indicate  that  students 
taught  to  troubleshoot  on  the  trainer  performed  as  well  during  practice  and  on  the  final 
performance  tests  as  did  students  taught  on  the  actual  equipment,  although  they  also 
suggest  that  students  preferred  use  of  the  actual  equipment  in  their  training.  This  result 
was  to  be  expected,  as  the  AN/WSC-3  is  a  relatively  simple  unit  to  troubleshoot,  and 
there  was  no  obvious  advantage  to  using  the  trainer.  Student  preference  can  therefore  be 
attributed  to  a  natural  desire  to  function  in  an  environment  as  close  to  the  real 
environment  as  possible.  In  other  circumstances,  where  GMTS  significantly  expedites 
training  or  provides  practice  not  possible  when  using  actual  equipment,  a  clear  advantage 
would  be  expected.  In  the  current  instance,  with  no  other  changes  to  the  curriculum,  the 
trainer's  availablity  shortened  the  course  from  4  to  3  weeks,  by  providing  8  systems  rather 
than  4  to  work  with  during  laboratory  periods. 

There  were  a  number  of  difficulties  in  the  conduct  of  the  research  that  significantly 
impacted  the  results.  The  Graf-pen  caused  a  number  of  problems,  but  they  were  minor 
compared  to  problems  with  the  microfiche  system,  which  was  difficult  to  align  and 
maintain.  Moreover,  the  carriers  holding  the  microfiche  had  to  be  replaced  frequently 
due  to  scratches  on  the  surfaces  that  created  disturbing  streaks  on  the  projection  screen. 
In  addition,  dust  accumulated  on  the  mirrors,  lens,  and  the  microfiche  itself,  creating 
large  "blobs"  on  the  projection  screen.  Changes  to  the  system  to  eliminate  these  faults 
during  the  research  period  would  have  required  extensive  modifications  that  were  beyond 
the  scope  of  the  current  task.  These  problems  did,  however,  lead  to  selection  of  a 
videodisc  for  image  presentation  and  touch  panels  for  input  for  the  EEMT  (Device  No. 
1  IB  106). 

During  the  course  of  the  research,  there  were  three  complete  turnovers  of  instructor 
personnel,  with  resulting  differences  in  instruction  and  test  monitoring.  In  addition,  there 
were  administrative  changes,  including  a  reduction  in  the  number  of  replacement 
opportunities  during  Experiments  2  and  3,  each  of  which  altered  the  climate  in  which  the 
research  was  conducted.  For  example,  the  reductions  in  completion  times  in  the  later 
stages  of  the  research  are  an  indication  of  increased  pressure  being  applied  to  the 
students  during  the  research. 

A  third  problem  was  with  errors  in  the  data  base.  Even  though  the  data  base 
developer  and  available  subject  matter  experts  (instructors)  reviewed  the  training  in 
detail  prior  to  the  start  of  the  research,  data  base  errors  kept  cropping  up  throughout  the 
experimental  period.  Student  use  uncovered  missing  images,  improper  test  equipment 
readings,  and  improperly  defined  touch  points.  While  these  errors  were  not  fatal  to  the 
simulation,  they  were  irritating  to  both  the  students  and  their  instructors.  Some  errors 


were  corrected  on  the  spot  or  in  very  short  order.  Some  errors  remained  throughout  the 
entire  experimental  period,  however,  producing  negative  feelings. 

In  spite  of  the  problems  that  impacted  the  final  results,  the  trainer  was  shown  to 
provide  effective  transfer.  Learning  with  the  trainer  required  approximately  the  same 
time  as  using  actual  equipment  and  transferred  successfully  to  use  of  actual  equipment. 
While  students  and  instructors  may  have  preferred  the  use  of  actual  equipment,  the 
trainer  simulator  provided  an  effective  way  to  supplement  training.  In  the  present 
instance,  it  reduced  overall  training  time  by  providing  more  units.  In  other  circum¬ 
stances,  it  could  provide  training  not  otherwise  available  to  students. 


CONCLUSIONS 

This  trio  of  experiments  demonstrated  that  the  GMTS  concept  can  be  an  effective 
alternative  to  the  use  of  actual  equipment  in  the  training  environment.  The  results 
suggest  that  the  EEMT,  which  incorporates  improvements  such  as  touch  panels  and  a 
videodisc  system  rather  than  the  Graf-pen  and  microfiche  projector,  should  be  even  more 
effective  than  the  GMTS  and  will  provide  meaningful  training  in  electronic  maintenance 
techniques  and  equipment  familiarization.  The  EEMT  can  provide  a  means  to  expedite 
training  and  to  reduce  training  costs.  Its  use  will  reduce  safety  hazards  to  personnel  and 
equipment  and  will  provide  opportunities  for  more  practice  in  those  situations  where  the 
availability  or  usability  of  actual  equipment  is  limited. 
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APPENDIX 

STUDY  QUESTIONNAIRES 

EXPERIMENTAL  GROUP  QUESTIONNAIRE  . 

CONTROL  GROUP  QUESTIONNAIRE  . . 

COMPARISON  QUESTIONNAIRE  . . 


A-0 


EXPERIMENTAL  GROUP  QUESTIONNAIRE 


1.  Name 


Student  if 


2.  Rating  _ 

3.  Number  of  years  experience  in  fleet  electronics 


4.  AESD  student 


Yes/No 


5.  Education  level 


fears  1-16 


6.  Are  you  a  graduate  of: 

a.  Naval  Electronics  "A"  School? 


b.  Naval  Electronics  "B"  School? 


c.  Approxmate  number  of  *'C"  Schools  attended? 


7.  Previous  experience  on  WSC-3  systems? 


Yes/No 


8.  Previous  experience  with  other  communications  equipment? 


Yes/No 


Yes/No 


Yes/No 


A- 1 


CIRCLE  THE  LETTER  IN  FRONT  OF  ONE  CHOICE  FOR  EACH  QUESTION 


1.  Compared  to  other  experiences  with  similar  subject  matter,  my  experience  with  the 
BTL  trainer  for  troubleshooting  practice  was: 

a.  Much  more  enjoyable. 

b.  More  enjoyable. 

c.  About  as  enjoyable. 

d.  Less  enjoyable. 

e.  Much  less  enjoyable. 

2.  As  a  technique  for  learning  and  practicing  troubleshooting,  the  BTL  trainer  was: 

a.  Very  effective. 

b.  Effective. 

c.  Both  effective  and  ineffective. 

d.  Somewhat  ineffective. 

e.  Very  ineffective. 

3.  The  BTL  trainer  was: 

a.  Very  easy  to  use. 

b.  Easy  to  use. 

c.  Neither  easy  nor  difficult  to  use. 

d.  Difficult  to  use. 

e.  Very  difficult  to  use. 

4.  The  initial  instruction  given  on  how  to  use  the  BTL  trainer  was: 

a.  Much  more  than  adequate  to  my  needs. 

b.  More  than  adequate  to  my  needs. 

c.  Adequate  to  my  needs. 

d.  Less  than  adequate  to  my  needs. 

e.  Much  less  than  adequate  to  my  needs. 

5.  It  took  me  the  following  time  to  learn  to  use  the  BTL  trainer: 

a.  About  1/2  hour  or  less. 

b.  About  1  hour. 

c.  About  1-1/2  hours. 

d.  About  2  hours. 

e.  More  than  2  hours. 

6.  Setting  switches  by  use  of  the  touch-pen  was: 

a.  Very  easy. 

b.  Easy. 

c.  Neither  easy  nor  difficult. 

d.  Difficult. 

e.  Very  difficult. 


A-2 


7.  Using  the  simulated  test  equipment  (oscilloscope  and  multimeter)  by  means  of  the 
touch-pen  was: 

a.  Very  easy. 

b.  Easy. 

c.  Neither  easy  nor  difficult. 

d.  Difficult. 

e.  Very  difficult. 

8.  The  microfiche  slides  were: 

a.  Very  easy  to  interpret. 

b.  Easy  to  interpret. 

c.  Both  easy  and  difficult  to  interpret. 

d.  Difficult  to  interpret. 

e.  Very  difficult  to  interpret. 

9.  The  symptoms  exhibited  by  the  BTL  trainer  for  the  malfunctions  were: 

a.  Very  accurate. 

b.  Generally  accurate. 

c.  Both  accurate  and  inaccurate. 

d.  Generally  inaccurate. 

e.  Very  inaccurate. 

10.  For  me,  the  problems  on  the  BTL  trainer  were: 

a.  Much  too  difficult. 

b.  Too  difficult. 

c.  About  the  correct  difficulty. 

d.  Too  easy. 

e.  Much  too  easy. 

1 1 .  The  amount  of  troubleshooting  practice  in  the  course  was: 

a.  Much  too  little. 

b.  Too  little. 

c.  About  the  right  amount. 

d.  More  than  necessary. 

e.  Much  more  than  necessary. 

12.  If  1  had  20  hours  to  practice  system  troubleshooting,  I  would  divide  my  time  as 
follows  between  the  BTL  trainer  and  the  actual  WSC-3  equipment: 

_  hours  on  the  BTL  trainer. 

_  hours  on  actual  equipment. 

1 3.  Prior  to  this  course  I  was: 

a.  Very  familiar  with  the  W5C-3. 

b.  Familiar  with  the  WSC-3. 

c.  Somewhat  familiar  with  the  WSC-3. 


d.  Not  very  familiar  with  the  WSC-3. 

e.  Not  at  all  familiar  with  the  WSC-3. 

We  are  interested  in  your  comments,  especially  if  you  have  ideas  or  criticisms  that 
not  conveyed  by  your  responses  to  the  questions  above. 


My  comments: 


CONTROL  GROUP  QUESTIONNAIRE 


1.  Name  Student  // 


2.  Rating  _ 

3.  Number  of  years  experience  in  fleet  electronics  _ _ _ 

4.  AESD  student 

Yes/No 

5.  Education  level _ 

Years  1-16 

6.  Are  you  a  graduate  of: 

a.  Naval  Electronics  "A"  School?  _ 

Yes/No 

b.  Naval  Electronics  "B"  School? 

Yes/No 

c.  Approxmate  number  of  "C"  Schools  attended?  _ 


7.  Previous  experience  on  WSC-3  systems?  _ 

Yes/No  Years 

8.  Previous  experience  with  other  communications  equipment?  _  _ 

Yes/No  Years 


CONTROL  GROUP  QUESTIONNAIRE 


1.  Compared  to  other  experiences  with  similar  subject  matter,  my  experience  with  the 
actual  WSC-3  equipment  for  troubleshooting  practice  was: 

a.  Much  more  enjoyable. 

b.  More  enjoyable. 

c.  About  as  enjoyable. 

d.  Less  enjoyable. 

e.  Much  less  enjoyable. 

2.  As  a  technique  for  learning  and  practicing  troubleshooting,  the  WSC-3  equipment 
was: 

a.  Very  effective. 

b.  Effective. 

c.  Both  effective  and  ineffective. 

d.  Somewhat  ineffective. 

e.  Very  ineffective. 

3.  The  WSC-3  equipment  was: 

a.  Very  easy  to  use. 

b.  Easy  to  use. 

c.  Neither  easy  nor  difficult  to  use. 

d.  Difficult  to  use. 

e.  Very  difficult  to  use. 

if.  Setting  switches  on  the  WSC-3  equipment  was: 

a.  Very  easy. 

b.  Easy. 

c.  Neither  easy  nor  difficult. 

d.  Difficult. 

e.  Very  difficult. 

5.  Using  the  test  equipment  (oscilliscope  and  multimeter)  was: 

a.  Very  easy. 

b.  Easy. 

c.  Neither  easy  nor  difficult. 

d.  Difficult. 

e.  Very  difficult. 

6.  For  me,  the  problems  on  the  WSC-3  equipment  were: 

a.  Much  too  difficult. 

b.  Too  difficult. 

c.  About  the  correct  difficulty. 

d.  Too  easy. 

e.  Much  too  easy. 


A-6 


7.  The  amount  of  troubleshooting  practice  in  the  course  was: 


a.  Much  too  little. 

b.  Too  little. 

c.  About  the  right  amount. 

d.  More  than  necessary. 

e.  Much  more  than  necessary. 

8.  Prior  to  this  course,  1  was: 

a.  Very  familiar  with  the  WSC-3. 

b.  Familiar  with  the  WSC-3. 

c.  Somewhat  familiar  with  the  WSC-3. 

d.  Not  very  familiar  with  the  WSC-3. 

e.  Not  at  all  familiar  with  the  WSC-3. 

We  are  interested  in  your  comments,  especially  if  you  have  ideas  or  criticisms  that  are 
not  conveyed  by  your  responses  to  the  questions  above. 


My  comments: 


COMPARISON  QUESTIONNAIRE 


1.  Name  _  Student  // _ 

2.  Rating _ 

3.  Number  of  years  experience  in  fleet  electronics  _ 

4.  AESD  student 

Yes/No 

5.  Education  level _ 

Years  1-16 

6.  Are  you  a  graduate  of: 

a.  Naval  Electronics  "A"  School? 

Yes/No 

b.  Naval  Electronics  "B"  School? 

Yes/No 

c.  Approximate  number  of  "C"  Schools  attended?  _ 


7.  Previous  experience  on  WSC-3  systems?  _ 

Yes/No  Years 

8.  Previous  experience  with  other  communications  equipment?  _____  _ 

Yes/No  Years 


A-9 


i 


CIRCLE  THE  LETTER  IN  FRONT  OF  ONE  CHOICE  FOR  EACH  QUESTION 


1.  Compared  to  other  training  experiences  with  similar  subject  matter,  my  experience 
with  the  BTL  trainer  for  troubleshooting  practice  was: 

a.  Much  more  enjoyable. 

b.  More  enjoyable. 

c.  About  as  enjoyable. 

d.  Less  enjoyable. 

e.  Much  less  enjoyable. 

2.  As  a  technique  for  learning  and  practicing  troubleshooting,  the  BTL  trainer  was: 

a.  Very  effective. 

b.  Effective. 

c.  Both  effective  and  ineffective. 

d.  Somewhat  ineffective. 

e.  Very  ineffective. 

3.  As  a  technique  for  learning  and  practicing  troubleshooting,  the  real  equipment  was: 

a.  Very  effective. 

b.  Effective. 

c.  Both  effective  and  ineffective. 

d.  Somewhat  ineffective. 

e.  Very  ineffective. 

4.  The  BTL  trainer  was: 

a.  Very  easy  to  use. 

b.  Easy  to  use. 

c.  Neither  easy  nor  difficult  to  use. 

d.  Difficulty  to  use. 

e.  Very  difficult  to  use. 

5.  The  initial  instruction  given  on  how  to  use  the  BTL  trainer  was: 

a.  Much  more  than  adequate  to  my  needs. 

b.  More  than  adequate  to  my  needs. 

c.  Adequate  to  my  needs. 

d.  Less  than  adequate  to  my  needs. 

e.  Much  less  than  adequate  to  my  needs. 

6.  It  took  me  the  following  time  to  learn  to  use  the  BTL  trainer: 

a.  About  1/2  hour  or  less. 

b.  About  1  hour. 

c.  About  1-1/2  hours. 

d.  About  2  hours. 

e.  More  than  2  hours. 


* 


* 


7.  Setting  switches  by  use  of  the  touch  pen  was: 

a.  Very  easy. 

b.  Easy. 

c.  Neither  easy  nor  difficult. 

d.  Difficult. 

e.  Very  difficult. 

8.  Using  the  simulated  equipment  (oscilloscope  and  multimeter)  by  means  of  the  touch- 
pen  was: 

a.  Very  easy. 

b.  Easy. 

c.  Neither  easy  nor  difficult. 

d.  Difficult. 

e.  Very  difficult. 

9.  The  microfiche  slides  were: 

a.  Very  easy  to  interpret. 

b.  Easy  to  interpret. 

c.  Both  easy  and  difficult  to  interpret. 

d.  Difficult  to  interpret. 

e.  Very  difficult  to  interpret. 

10.  The  symptoms  exhibited  by  the  BTL  trainer  for  the  malfunction  were: 

a.  Very  accurate. 

b.  Generally  accurate. 

c.  Both  accurate  and  inaccurate. 

d.  Generally  inaccurate. 

e.  Very  inaccurate. 

1 1.  For  me,  the  problems  on  the  BTL  trainer  were: 

a.  Much  too  difficult. 

b.  Too  difficult. 

c.  About  the  correct  difficulty. 

d.  Too  easy. 

e.  Much  to  easy. 

12.  The  amount  of  troubleshooting  practice  in  the  course  was: 

a.  Much  too  little. 

b.  Too  little. 

c.  About  the  right  amount. 

d.  More  than  necessary. 

e.  Much  more  than  necessary. 


I 

I 
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3.  If  I  had  20  hours  to  practice  system  troubleshooting,  I  would  divide  my  time  as 
follows  between  the  BTL  trainer  and  the  actual  WSC-3  equipment: 

_  hours  on  the  BTL  trainer. 

_  hours  on  actual  equipment. 

4.  Prior  to  this  course  I  was: 

a.  Very  familiar  with  the  WSC-3. 

b.  Familiar  with  the  WSC-3. 

c.  Somewhat  familiar  with  the  WSC-3. 

d.  Not  very  familiar  with  the  WSC-3. 

e.  Not  at  all  familiar  with  the  WSC-3. 

We  are  interested  in  your  comments,  especially  if  you  have  ideas  or  criticisms  that  are 
not  conveyed  by  your  responses  to  the  questions  above. 


My  comments: 
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